UW CSE Technical Report 03-06-02 The Trajectory Mixture Model for Learning Collections of Nonlinear Functions

Learning statistical models of nonlinear dynamical systems has long been an important problem in machine learning. The problem becomes especially hard when the dynamical system is composed of a mixture of nonlinear models, not just a single nonlinear model. To address this general case of nonlinear time-series modeling, we propose a new hierarchical architecture: the Trajectory Mixture Model (TMM). The TMM learns collections of different nonlinear “trajectories” through state space. The model uses an expectation maximization (EM) algorithm to train a collection of nonlinear function approximators based on Gaussian radial basis function units. State densities are represented using samples estimated by particle filtering and smoothing. A sample-based representation provides an effective means of representing non-parametric state densities that change arbitrarily over time. We use entropy-based model selection to ensure that the individual function approximators, as well as the higherlevel mixture model, do not overfit the data. Our results suggest that TMMs can learn complex nonlinear state space models directly from observations and may offer greater flexibility in modeling time-series data than existing methods such as extended Kalman filters and particle filters.